Defining the Enablers and Barriers to the Implementation of Large-scale, Health Care-Related Mobile Technology: Qualitative Case Study in a Tertiary Hospital Setting.

BACKGROUND: The successful implementation of clinical smartphone apps in hospital settings requires close collaboration with industry partners. A large-scale, hospital-wide implementation of a clinical mobile app for health care professionals developed in partnership with Google Health and academic partners was deployed on a bring-your-own-device basis using mobile device management at our UK academic hospital. As this was the first large-scale implementation of this type of innovation in the UK health system, important insights and lessons learned from the deployment may be useful to other organizations considering implementing similar technology in partnership with commercial companies. OBJECTIVE: The aims of this study are to define the key enablers and barriers and to propose a road map for the implementation of a hospital-wide clinical mobile app developed in collaboration with an industry partner as a data processor and an academic partner for independent evaluation. METHODS: Semistructured interviews were conducted with high-level stakeholders from industry, academia, and health care providers who had instrumental roles in the implementation of the app at our hospital. The interviews explored the participants' views on the enablers and barriers to the implementation process. The interviews were analyzed using a broadly deductive approach to thematic analysis. RESULTS: In total, 14 participants were interviewed. Key enablers identified were the establishment of a steering committee with high-level clinical involvement, well-defined roles and responsibilities between partners, effective communication strategies with end users, safe information governance precautions, and increased patient engagement and transparency. Barriers identified were the lack of dedicated resources for mobile change at our hospital, risk aversion, unclear strategy and regulation, and the implications of bring-your-own-device and mobile device management policies. The key lessons learned from the deployment process were highlighted, and a road map for the implementation of large-scale clinical mobile apps in hospital settings was proposed. CONCLUSIONS: Despite partnering with one of the world's biggest technology companies, the cultural and technological change required for mobile working and implementation in health care was found to be a significant challenge. With an increasing requirement for health care organizations to partner with industry for advanced mobile technologies, the lessons learned from our implementation can influence how other health care organizations undertake a similar mobile change and improve the chances of successful widespread mobile transformation.

HypoML: Visual Analysis for Hypothesis-based Evaluation of Machine Learning Models.

In this paper, we present a visual analytics tool for enabling hypothesis-based evaluation of machine learning (ML) models. We describe a novel ML-testing framework that combines the traditional statistical hypothesis testing (commonly used in empirical research) with logical reasoning about the conclusions of multiple hypotheses. The framework defines a controlled configuration for testing a number of hypotheses as to whether and how some extra information about a "concept" or "feature" may benefit or hinder an ML model. Because reasoning multiple hypotheses is not always straightforward, we provide HypoML as a visual analysis tool, with which, the multi-thread testing results are first transformed to analytical results using statistical and logical inferences, and then to a visual representation for rapid observation of the conclusions and the logical flow between the testing results and hypotheses. We have applied HypoML to a number of hypothesized concepts, demonstrating the intuitive and explainable nature of the visual analysis.